So I’ve been in the AWS cloud space for a long time which has been great as they really have phenomenal cloud offering, but in that time Kubernetes has been steadily gaining speed as an alternative hybrid approach to cloud computing. And while I’ve read some things and worked on containers running in K8’s I’ve never really had an in depth understanding of the cluster management system. So I decided to fix that by doing the "Kubernetes the Hard Way" tutorial. It’s been great, while not a whole lot of it is new, it really is great practice and a very good end to end, "secure", setup walk through. So these are some take a ways I have from the experience.

While the tutorial says "no scripts", that’s not exactly accurate. It will have you write your own setup scripts and bash is the language used to explain those operations. While the code display pieces on the tutorial have an "easy button" to copy the code and paste it in your terminal there are two main reasons not to do this:

1. Don’t copy and paste code from the web in your terminal! There are well documented attack vectors that can compromise your entire system by doing this.
2. The idea is to get a non-shallow understanding of Kubernetes, copying the code and writing it your way adds muscle memory to the exercise.

Additionally unless you are a super human typist there are the inevitable typos, bugs and such that actually help to learn in depth how to diagnose problems in Kubernetes and fix them. This is really what I find most useful about doing it the hard way to really learn. For instance, the tutorial assumes using tmux in parallel manner across 3 controllers when initializing etcd on the controllers, which I didn’t do. Through which I found out that the second controller must be initialized within the timeout period of the daemon start on the first or you will get a timeout error (after those first two are initialized this is no longer an issue). I would likely have never learned this by using scripted setups for K8’s clusters.

Or like when I typo’d a cert location as /etc/kubernetes.pem instead of /etc/etcd/kubernetes.pem and I learned I’ve been absolutely spoiled by Rust’s detailed and helpful compiler error messages. The error message was something like ERROR: systemd returned a status of "error code" returned. I know what you are thinking, "error code" should be more than enough for you to know what went wrong. Unfortunately I needed a bit more detail to figure out the problem so a bit of research showed me the command

journalctl -ocat -b -u etcd

Where "etcd" is really whatever your systemd daemon name is. I think I’m going to alias this to doh in my shell for future reference. I know journalctl but the argument soup is a super useful combo for working with systemd daemons, but one which I’m not sure I’ve ever used or have forgotten. So learning/re-learning it because I’m doing this the "hard way" has been really great. I’d highly recommend this tutorial if you’d like to learn hands on about Kubernetes.

Kubernetes the Hard Way